feat(bedrock): add TTL support to auto-injected tool and system/user cache points#2232
Open
kpx-dev wants to merge 1 commit intostrands-agents:mainfrom
Open
feat(bedrock): add TTL support to auto-injected tool and system/user cache points#2232kpx-dev wants to merge 1 commit intostrands-agents:mainfrom
kpx-dev wants to merge 1 commit intostrands-agents:mainfrom
Conversation
Open
3 tasks
Contributor
Author
|
Added 3 integration tests covering the new paths. Ran locally against Coverage matrix:
|
…cache points Extends prompt caching TTL coverage beyond user-supplied cachePoint blocks (PR strands-agents#1660) to the two SDK-managed auto-injected paths on BedrockModel: - Adds cache_tools_ttl config option so the toolConfig auto-injected cache point can carry a TTL (e.g. '5m' or '1h'). - Adds ttl field to CacheConfig dataclass so _inject_cache_point propagates TTL into the cache point appended to the last user message when strategy='auto'. Together, these let users align all three cache checkpoint TTLs (toolConfig -> system -> messages) to satisfy Bedrock's non-increasing TTL ordering rule -- which was previously impossible because cache_tools hardcoded an implicit 5m TTL. Partially addresses strands-agents#2121 (Bug 2: cache_tools ordering violation with 1h TTL). Bug 1 from strands-agents#2121 was resolved by strands-agents#1660. Tests: - 4 unit tests covering cache_tools_ttl and CacheConfig.ttl with and without TTL (backward-compat). - 3 integration tests against Claude Haiku 4.5 (officially documented for 1h TTL on Bedrock), including a regression test that sets 1h TTL on all three cache checkpoints simultaneously. - Model ID extracted into a _CACHE_TTL_MODEL_ID module constant so future model bumps are a one-line change.
6a74d89 to
797f359
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Extends prompt caching TTL coverage beyond the user-supplied
cachePointpath (already fixed in #1660) to the two SDK-managed auto-injected paths onBedrockModel.Previously, only cache points that users passed directly inside message content could carry a
ttlfield — the formatter preserved it end-to-end. However, when the SDK auto-injected cache points on the user's behalf, thettlwas hardcoded away:cache_tools—BedrockModel(cache_tools="default")appends{"cachePoint": {"type": "default"}}totoolConfig.toolswith no way to set a TTL._inject_cache_point— whencache_config=CacheConfig(strategy="auto")is set, the SDK appends{"cachePoint": {"type": "default"}}to the last user message with no way to set a TTL.Because Bedrock processes cache checkpoints in order
toolConfig → system → messagesand requires TTLs to be non-increasing, a user who wanted to run 1h caching end-to-end could not — the SDK-managed checkpoints would implicitly be 5m and violate the ordering rule (see #2121 Bug 2).Changes
BedrockModel: addedcache_tools_ttl: str | Noneconfig option. When set together withcache_tools, the TTL is propagated into thetoolConfigcache point.CacheConfig: addedttl: str | Nonefield. When set,_inject_cache_pointincludes the TTL in the cache point appended to the last user message.Users can now align all three cache checkpoint TTLs consistently:
API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_CachePointBlock.html
Launch announcement: https://aws.amazon.com/about-aws/whats-new/2026/01/amazon-bedrock-one-hour-duration-prompt-caching/
Related Issues
Partially addresses #2121 (Bug 2:
cache_toolsordering violation with 1h TTL).Bug 1 from #2121 (user-supplied
ttlbeing dropped) was resolved by #1660.Documentation PR
Type of Change
New feature
Testing
Added 4 unit tests:
test_inject_cache_point_with_ttl— verifiesCacheConfig(ttl="5m")propagates into the auto-injected message cache point.test_inject_cache_point_without_ttl— backward compat:CacheConfig(strategy="auto")with no TTL still works.test_format_request_cache_tools_with_ttl— verifiescache_tools_ttlpropagates intotoolConfigcache point.test_format_request_cache_tools_without_ttl— backward compat:cache_toolswithout TTL still works.Verified:
hatch run prepare(lint, format, mypy all clean; pre-existing local env test failures unrelated to this change)Checklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.